LRTDP Versus UCT for Online Probabilistic Planning

UCT, the premier method for solving games such as Go, is also becoming dominant algorithm probabilistic planning. Out of five solvers at International Probabilistic Planning Competition (IPPC) 2011, four were based on UCT algorithm. However, while a UCT-based planner, PROST, won contest, an LRTDP-based system, Glutton, came in close second, outperforming other systems derived from UCT. These results raise question: what are strengths and weaknesses LRTDP practice? This paper starts answering this question by contrasting two approaches context finite-horizon MDPs. We demonstrate that scenarios, UCT's lack sound termination condition serious practical disadvantage. In order to handle MDP with large finite horizon under time constraint, forces expert guess non-myopic lookahead value which it should be able converge encountered states. Mistakes setting parameter can greatly hurt performance. contrast, LRTDP's convergence criterion allows iterative deepening strategy. Using strategy, automatically finds largest feasible given constraint. As result, has better performance stronger theoretical properties. present online version named Gourmand, illustrates analysis outperforms PROST set IPPC-2011 problems.